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Introduction 

Educational research often involves collecting data that have a nested sampling 
structure. For example students are randomly selected from a random selection of 
schools. As well, educational research often addresses questions that are at the 
institutional level while data are collected only at the student level. For example, 
studies on school culture were often based on questionnaire data collected from 
students. It has been established by recent developments in research methods that 
hierarchical structural models are to be preferred over conventional data, or in designs 
involving research questions at the institutional levels. (See for example, Aitkin & 
Longford, 1986; Bock, 1989; Bryk & Raudenbush, 1992; Goldstein, 1987; Mason, 
Wong & Entwisle, 1983; Raudenbush & Bryk, 1985.) 

In the area of structural equation models, discussions emanating from Goldstein & 
McDonald's (1988) general model for bilevel structures include developments on the 
statistical theory (Lee, 1990) and validations of the model on empirical data sets (e.g. 
Muthen, 1989; 1991). 

Nevertheless, the literature is still unclear as to the selection of approprir te multilevel 
factor models from those resulting from (i) decisions on scaling at the various levels, 
and (ii) the possibility of equating or not equating factor loadings across levels. It is 
the aim of this paper to (a) discuss the characteristics of seven bilevel factor models, 
and (b) arrive at a list of empirically based recommendations for selecting appropriate 
bilevel factor models, the assumptions underlying these decisions, and the 
consequences for interpretation for the Spearman Case. 

Theoretical Framework 

The models discussed in this paper are special cases of a bilevel model given by 
McDonald and Goldstein (1989), in which measures are obtained at the student level 
only. A q x 1 data vector of measures on student i randomly sampled from school 

j , also randomly sampled, t/* (i =1, n j , ; =1, m) is given a variance 
component decomposition 



ya = M- + y(*)i + ymi 



ilamitk* Kf^ Fyt/v UMafc Pip 0 nil g 



where ji is the grand mean, yw, is the between - school component and yw» is the 

within - school component The bilevel factor model is, using results from McDonald 
and Goldstein (1989), 

y* = XjZ 9 + u 9 

or 

y = A z+ u 

where z is the factor score, u is the unique component and A is the factor loading 
matrix, and is given by 



A = ®(/*®A./> 

The covariance matrix, ^ of y is decomposed into between - and within - school 
components and respectively, as 



such that 



5^ =AAA , .+ »P, 

where the A's are the factor loading matrices, 0's are factor covariance matrices and 
^Fs are diagonal residual covariance matrices, with the subscripts indicating the 
corresponding levels. By equating or nor equating A B and A w , and by choosing 
various scaling factors on 0, and different structural models, some of which may 
be shown to be reparameterizations of others, can be fitted. These are discussed in 
more detail below, in the context of a Spearman Case. 
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I, Model A 



In this model, the original matrix of measurements is used and the factor loadings are 
constrained equal across the two levels. By decomposing the factor score and the 
unique factor of the data vector into a between - school and a within - school 
component, this model can be represented by: 



y,=X(TV+C')+(V/ + ef) 

=(?CTV+V/)+(XC*+e,) 

where T| and £ are the school-level and student-level factor scores and v and e are 
the school- and student-level unique factor respectively. 

The corresponding covariance matrix ^ is then decomposed into the between - 
and within - school components: 

where 2, 



and (j^ is the between - school factor variance. 

This model has the simple interpretation that there is between - school variability and 
within - school variability in the common factor and the unique factors, yielding the 
corresponding between / within variability in the item scores. In this model, the 

between - school factor variance is direcdy estimated. The ratio q\ I (0* + 1) 
provides a natural estimate of school effects. The limitation of this model is the 
constraint on the equality of factor loadings across levels, with the implication that 
the same factor underlies the behaviour of items at the school- and the student - 
levels. These points can be elaborated using a six - item scale designed to measure 
the quality of the formal curriculum (QUAL). The items in this scale are 

(1) The curriculum of this school meets my present needs. 

(2) This school offers a good range of subjects in Years 1 1 and 12. 

(3) The subjects offered at this school develop the capacity for independent and 
critical thinking. 

(4) The subjects taught in this school offer useful knowledge or skills. 
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(5) The subjects taught in this school are relevant to real life and to students 1 needs. 

(6) The subjects taught here prepare students adequately for future employment. 

The semantics of these items suggest that item 2 refers more to the curriculum at the 
institutional level while item 1 is more related to the students 1 personal opinion of the 
school curriculum. The rest of the items in the scale are expected to be impacted 
upon by a mixture of school-level and student-level variations. As such, the 
constrained-equal loading model might not be the best to use. Schematically, model 
A for the QUALity of the formal Curriculum Scale can be represented by Figure 1 
below. 



Place Figure 1 about here 



Relaxing the equal loading constraint leads naturally to the next model below. 



n. Model B 

This model relaxes the assumption on equal loadings across levels of model A. The 
previous metric is still used in the analysis. This model, using the same notation as 
before, is: 



=(A, .TV + v}+(A, 



where 

The price for allowing factor loadings to freely vary across the two levels is that, in 
the usual sense in single-level factor analysis, factor variances at both levels need to 
be fixed at unity for identifiability. The consequence is that there is no direct way of 
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estimating school effects parallel to that in Model A. It is possible, on the other hand, 
to compute the ratio of 

/ / / 

X «X • / (X «A,«+ X wXw) 

as an estimate of the relative amount of information contributed by the school-level 
variations. Nevertheless, the more severe consequence of this distinct loading model 
is the implication that, in cases where the loadings are significantly different across 
levels, the factors at the school- and student-level are different. In other words, the 
latent attribute of the school as registered through students' perceptions may have a 
different interpretation to Uie latent attribute of the student registered through tin 
same group of students with the same set of measuring instruments. Just for the sake 
of argument, suppose the school-level loadings in the Quality of formal school 
curriculum were "defined by" items 2, 3, 4 such that the school-level factor was one 
of quality in terms of subject range and levels of academic content, while student- 
level loadings suggested a factor defined by how well student needs were met by the 
curriculum. The factors measured by these items would then be completely different 
traits at the school- and student - levels, It would therefore be misleading to label both 
with the same factor name and any computation of proportion of variance explained 
by school versus by student would be erroneous. Of course the caveat extends itself 
naturally to the possibility of different factor structures in terms of number of factors 
across the two levels. Model B is represented schematically by Fig. 2. 



Place Figure 2 about here 



The above arguments are all very well except for a problem with the scaling of the 
units of measurement. In the same sense in which we are not usually willing to 
compare unstandardized regression weights, if the spread of the indicators of a factor 
are different, it is not possible to compare relative contributions of these indicators in 
defining the factor by comparing the factor loadings; without appropriate scaling, the 
loadings represent the weights of these indicators measured in their natural units. 
This necessitates the development of the next set of models. 

ITL Model C 

The variables are scaled to yield standardized factor loadings at both levels first for 
this model. Like Model A, the loadings in Model C are constrained equal across 
levels. In single-level factor analysis, scale invariant models are obtained by pre- and 
post- multiplication of a structured correlation matrix by a non - singular diagonal 
scaling matrix, A, which is to be estimated (Krane & McDonald, 1978. See also 
Brown, 1982; McDonald, Parker & Ishizuka, in press). 

The mathematical expressions defining Model C are: 
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where ^ and are the scaling matrices at the between - and within - levels 
respectively and are to be estimated. Extending the arguments for the single-level 

case given in McDonald, Parker, and Ishizuka (in press) to the bilevel situation, V w 
and ]j£ w are both constrained such that Diag {Pb)=I and Diag {Pw}=* » in 

X, =A.!P.A. «*» E w = A*fP*A„ 

by 

(P 9 -aa + %{/-aa } 

yielding standardized factor loadings jx. This model is to be preferred over Model A 
whenever the variance of the indicators are very different. However because the 
equated loadings in this Model are derived from standardised scores, comparisons can 
only be made in the relative sense. The constraint on holding the common factor 
variances at unity at both levels implies that the general size of school effects cannot 
be direcdy defined and estimated. 

HL Mndd n 

This model is an attempt to relax the restrictions on Model C such that school-level 
factor variance can be estimated with scaling for standardized parameters at both 
levels. Model D is thus a model involving scaling at both levels and constraining 
loadings across levels. 

This model is represented by: 



ywt = A«£Wf = A<* (a^+eO 

and 



s 



/tfamtffai RJhvil FyJ ^ Pa y> 7 of is 



E,=A,(ac 2 y+c, >A* 

X„=Aw(aa'+C* )A. 

where £ a and are diagonal matrices of unique variances satisfying the scaling 
constraints. Model D offers a direct estimate of school-level variance, viz, (5^ and 

the corresponding school effect estimate of (J* / ((J* + 1). Estimates of school- 
level loadings tend to be more stable (smaller SE) compared to the distinct loadings 
counter parts because estimation is based on the pooled larger sample size at the 
student- level. In research situations where distinct loadings are expected across 
levels, the next model may be considered. 

V, Model E 

Model E is an attempt to relax the constraints on equal loadings across levels while 
still standardising the variables at both levels. Mathematically, this model is 
represented by: 

ywt = A*#wi = A* (cc-ti+v,) 

and 

X«r= A*(aV"+C*)A* 

This model can be shown to be a reparameterization of Model B by writing 

^• = A*«- 

and L=^a* 

This model then is expected to give the same fit to the data as Model B and all the 
comments regarding Model B apply here. The additional advantage of Model E over 
Model B is that by means of the standardisation procedure, the loadings can be 
interpreted by the usual rules for standardized parameters. 
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VI. Other Bilevel Fnctnr Models 

Mathematically it is possible to scale for standardized loadings only at the student- 
level while allowing the school-level to remain at the original units of measurement if 
the loadings are distinct across levels: 



with 
and 

Indeed by writing 



this can be shown to be a reparameterization of Model E. Nevertheless, there does 
not seem to be an empirical situation which motivates this type of "mid - way" 
scaling. So this model will be treated as one with theoretical interest only but with 
very little use in terms of applications. 

On the other hand, it is also possible to fit a mathematical counter part for the "Equal- 
Loadings-Scaling-at-Student-Level-only" model. Again, this model does not seem 
well motivated. 

So far decompositions in all the models discussed are in terms of common factor 
scores and unique factor scores, with all parameters fixed. It is perfectly legitimate 
on the other hand to decompose the factor loadings into between - and within - 
components in a manner analogous to the random coefficients models discussed by de 
Leeuw and Kreft (1986). That is, 



y =(a +(J WH 
=(a + p K+(v +e ) 

-0 -/ 9 ~j -* 
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For the Quality of Formal Curriculum scale, this model can be represented by Figure 
3. 



Place Figure 3 about here 



In other words, the random coefficient model implies that the loadings of the manifest 
variables on the common factor are being randomly "modified" by school 
memberships. In this model, students are assumed to have a perception of the quality 
of school curriculum and that this perception is being coloured by the particular 
school the student happen to belong to. School effect is modelled through its impact 
on the relationship between the latent and manifest variables. The software to be 
reported in this paper was not written to handle a random coefficient model and so 
this model will not be further discussed. Nevertheless, it is possible to give an 
estimate of the profile of the slopes simply by repeatedly running a single level factor 
analysis on each school sample if needed. 

Source of Data 

Illustrative data consist of responses to a questionnaire from 5,932 Year 12 students. 
The students were selected from 50 Catholic schools from the Dioceses of NSW, 
Australia, by means of stratified random sampling (Flynn, 1992). The questionnaire 
contains 46 Likert - type items pertaining to 6 scales which were constructed to 
measure school culture. The scales are: Quality of Formal School Curriculum 
(QUAL, 6 items), Out - of - school curriculum (OUTS, 4 items), Relationships with 
Teachers (ATEC, 12 items), Student Morale (STDM, 10 items), Attitudes to the 
Principal (PRIN, 5 items), and the Attitudes to Discipline (ATTD, 4 items). The 
rationale behind these scales is discussed elsewhere (Flynn, 1992; Mok, 1992) and 
will not be repeated here. 

Methods 

The strategies (with modification) for multilevel covariance structure analysis 
recommended by Muthen (1991) were adopted. The steps were: 



Step 1. Conventional Factor Analysis nf £j 

An Maximum Likelihood confirmatory factor analysis of each of the six scales was 
conducted by means by the COSAN computer package (Fraser, 1987). Items which 
have a loading less than 0.33 were removed if necessary. This step was conducted 
with the understanding that the analysis was incorrect in the sense that the nested 
nature of the data had been ignored (Muthen, 1991) 



ERLC 



11 



Alflndw Bliwl Fyfr Ucm ?V* 1fl.nl.lfi 



Step 2. Estimation ofBetween Variation 

The intra - class correlation of each item in each scale was computed by means of 
fitting a variance components model to the data. The intra - class correlation gives 
the proportion of variance accounted for by the school. If it were small then 
multilevel analysis might not be warranted (Muthen, 1991). This step therefore was 
aimed to determine the legitimacy of the bilevel factor analysis to follow. The 
variance component model for each item was fitted by the ML3 computer software 
(Rasbash, Prosser, & Goldstein, 1990), which adopted an Iterative Generalised Least 
Squares algorithm in minimising the log - likelihood function (Goldstein, 1989). 

Step 3. Bilevel Factor Analysis 

The analyses reported here were performed using McDonald's BIRAM (Bilevel 
Reticular Action Model) software program (McDonald, Lam, Middlehurst, & Parker, 
in preparation). This program is written to implement the Reticular Action Model 
(McArdle & McDonald, 1984) for bilevel path analysis. Maximum Likelihood 
estimates of parameters are obtained by a quasi - Newton iterative procedure. 
Goodness of fit is obtained by comparing the negative log - likelihood ratios (chi - 
squared values) of the fitted model to a saturated model. The program provides the 
option for the user to set up the model for the run, or to use some existing models 
from previous runs. A control file was set up for each of the models discussed 
(Model A to E) for each of the six school culture scales. This was done by (i) 
selecting either scaling for standardized parameters (at both levels) or no scaling and 
(ii) specifying either estimating or not estimating school-level factor variances. 

For each scale and within each model, factor loadings across school- and student - 
levels were compared. Factor loadings between models for each level were compared 
and interpreted. Goodness of fit (Chi-squared value) across the models on each of the 
scales was also compared and contrasted as well. As BIRAM is a program still under 
development, it is anticipated that difficulties associated with the convergence of the 
minimisation procedure as a consequence of the combination of small intraclass 
correlation and unbalanced sample sizes would occur for some of the scales. These 
difficulties were also summarised here for further understanding of the selection of 
bilevel factor models. Altogether 5 x 6 - 30 BIRAM analyses were performed. 

The BIRAM program also has a provision for running a pseudo balanced analysis 
whereby all second level units (schools) were assumed to take an average sample size 
of first level units (students), or an unbalanced analysis whereby the actual sample 
size is used in the estimation, taking the results from the pseudo balanced run as 
initial values for the minimisation procedure. Where-ever available Chi-squared 
values from both pseudo balanced and unbalanced runs were reported here. The aim 
was to compare the goodness of fit between pseudo balanced and unbalanced runs for 
a badly unbalanced situation where school sizes ranged from 55 to 348. 
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Results 

Step 1. Results of the Conventional Factor An alysis Of Sj 

Conventional Factor Analysis using the COS AN software package (Fraser, 1987) on 
each of the six scales identified items not contributing to the definition of the factor. 
These items were inspected and subsequently removed. The remaining items were 
reanalysed using COS AN and the results are summarised in Table 1. 



Place Table 1 about here 



The results from this step indicated reasonable fit for all of the scales with chi- 
squared values ranging from 2770.3 (df = 54) to 24.13 (df = 2) with the 
corresponding McDonald's Goodness of fit Indices ranging from 0.80 to 0.998 
(McDonald, 1989). The Root Mean Square residual covariances ranged from 0.01 to 
0.05 . The Cronbach's Alpha as a measure of internal consistency for these scale 
ranged from 0.67 to 0.91. 

Step 2. Results on Estimation of Between Variation 

Intraclass correlations of the 41 items identified from the previous step ranged from 
0.08 to 0.18 (Table 1). These figures concurred with the other studies reported on 
school effectiveness (Reynolds, 1992). Muthen (1991) has suggested that if intraclass 
correlations are too small, multilevel analysis is not warranted. Intra - class 
correlations in studies reported by Muthen (1991) are a lot higher . It would be 
interesting to compare the effects of the sizes of the intraclass correlation on the 
different models. This is done in a simulation study by McDonald and Mok (1993). 

Step 3. Results of Bilevel Factor Analysis 

The hostile characteristic of this data set with a combination of small intra - class 
correlations, extremely unbalanced school sizes, and a large sample size at the 
student-level puts the BIRAM program to a severe test. The dimensions of 
intermediate computational stages exceeded the current limits of the BIRAM program 
for the two longer (with more than 10 items) scales (Relationships with teachers, 12 
items, and Student Morale, 10 items). Even for the remaining four shorter scales 
(with less than 10 items), some runs failed to give standard error estimates. The 
results of 20 BIRAM runs for these four scales are summarised in Table 2. 



Place Table 2 about here 



Table 2 contains the factor loadings at the school- and student - levels and factor 
variance estimates where appropriate for each scale. Referring to the factor loadings 
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for the Quality of Formal Curriculum Scale (QUAL), which has a low intra class 
correlation of 0.08, an item-to-item comparison between the values of the student- 
level loadings and the single-level loadings showed very little difference between 
them. This was especially true for models involving scaling* On the other hand, for 
the Out - of - School Curriculum (OUTS), Attitudes toward the Principal (PRIN), and 
the Attitudes toward Discipline (ATTD) scales, where the intra-class correlations 
were higher, more discrepancies between student-level loadings from the various 
bilevel factor models and those obtained from single-level factor analysis were 
observed. This was especially the case for models without scaling. These 
observations implied that if the intra class correlation is small, and if the research 
question involves a concept at the student-level only (e.g. quality of school life from 
the students' experience) then the conventional single-level factor analysis might be 
considered sufficient. On the other hand, if the intra - class correlation is high, then it 
might be worthwhile to proceed to the bilevel factor models which provide better 
estimates over the single-level factor analysis. 

Comparisons of school-level loadings with student-level loadings were, strictly 
speaking, only legitimate on Model E which involved scaling at both levels and 
which allowed distinct loadings across levels. For all the scales that had successful 
runs on BIRAM, school-level loadings were higher than student-level loadings after 
scaling at both levels. The proportion of information at the school-level was 
estimated to be 65% for the Quality of Formal Curriculum Scale (QUAL), 62% for 
the Out-of-School Curriculum Scale (OUTS), 61% for the Attitudes toward the 
Principal Scale (PRIN), and 70% for the Attitudes towards Discipline Scale (ATTD) 
(Table 2). It is unfortunate that both the Out-of-School Curriculum and the Attitudes 
towards Discipline scales, each with an intra class correlation of 0.16, involved 
Heywood cases (Heywood, 1931) on Model E. Muthen (1991) has warned that 
Heywood cases were as common in Bilevel factor analysis as in single level factor 
analysis. 

Referring to Model E again, school-level factor loadings for all the items in the 
Quality of Formal Curriculum Scale (QUAL) were in the older of 0.9 while loadings 
at the student-level were on the order of 0.6 to 0.7. However, the within item ratio of 
the school - to student - levels loadings were not uniform across all items, some items 
had relatively higher loadings at the school-level than others had. The same remark 
can be made for die Attitudes towards the Principal Scale (PRIN). School-level 
loadings for items 2, 3, 5 were relatively higher than those at the student-level, 
whereas school- and student-level loadings for items 1 and 4 were similar. These 
made good conceptual sense on inspection of the items (see Table 1). Items 2, 3, 5 
involved the leadership of the principal while items 1 and 4 were respectively 
involved with how approachable the principal was perceived and the importance 
placed by the principal on the religious nature of the school. 

The size of factor loadings from the unsealed models (Models A and B) might just be 
a reflection of the items' variabilities. Their interpretations can be as misleading as 
comparing raw regression coefficients. However, Model A (unconstrained and with 
no scaling) provided direct estimates of school-level factor variances, which ranged 
from 7% to 23% (Table 2). The size of the school-level factor variance tends to 
associate with the size of intra class correlation, but there does not seem to be a cne- 
to-one relationship between them. 
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The Chi-squared goodness of fit values of all models on all scales for both the pseudo 
balanced and the unbalanced runs are also given in Table 2. It is possible to make 
pairwise comparisons between models on goodness of fit for each individual scale 
using information presented in Table 3. For example, for the Quality of Formal 
Curriculum (QUAL), bilevel factor model A gains 14 degrees of freedom at a cost of 
losing 2.01 points on Chi - squared value of goodness of fit over the single-level 
analysis. The loss was not statistically significant at 5% level, and because in 
addition the research question involved one at the school-level, the move from the 
single-level factor analysis to Model A of the bilevel factor analysis was justified. 
The exercise of such pairwise comparisons across all models and all scales suggests 
that at least for the scales reported here, (i) if the loadings were unconstrained across 
levels, then scaling and non-scaling give the same goodness of fit: the models (B and 
E) are reparameterizations of one another, (ii) the unconstrained models (B and E) are 
not significantly better than constrained Model D which involves scaling at both 
levels and which requests school-level factor variance to be estimated. So the more 
parsimonous Model D is to be preferred over the other models; (iii) each of Models 
B, E, and D is significantly better than the constrained Model A which involves no 
scaling; (iv) each of these four models is better than Model C which is in turn not 
significantly different from the single-level factor analysis in terms of goodness of fit 
All bilevel factor models except Model C yield better fit to the data than the 
conventional factor model. 

The Chi - squared values for the pseudo - balanced and the unbalanced runs were next 
compared. All the Chi - squared values from the pseudo - balanced runs were larger 
than the unbalanced runs indicating a better fit by the latter. However, only 3 of the 
20 differences were larger than 7.88. The values of the loadings show that even with 
an unbalanced data set which had sample size ranging from 55 to 348, 
pseudobalanced estimates approximate the unbalanced satisfactorily. 

Conclusions 

The questions facing the researcher in decisions regarding bilevel factor analysis 
models are : 

(a) Whether bilevel factor analysis is required? 

(b) If a bilevel factor analysis is to be performed what are the available models? 

(c) How do we choose among these models? 

Various authors have written on the first question. The general advice hinges on two 
points: (i) whether the research question is conceptualised at the student-level (eg 
quality of school life as experienced by the student), or at the school-level (eg culture 
of the school as an institution), and (ii) whether there is enough between - variation to 
warrant multilevel analysis. 

Once a decision is made to conduct a multilevel analysis, it is necessary for the 
researcher to consider the class of alternative models. 

This paper discusses some of the available alternative bilevel factor models. Eight 
bilevel factor models were proposed, five of them were discussed in greater detail just 
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for the Spearman case. Selections of these models are discussed in the context of a 
set of empirical data on school culture. The five models are 

(A) no scaling, loadings constrained equal across levels, 

(B) no scaling, distinct loadings permitted, 

(C) scaling at both levels, constrained equal loadings, 

(D) scaling at both levels, constrained equal loadings, 
estimating school-level factor variance, 

(E) scaling at both levels, distinct loadings. 

Analyses in this study suggest that choices amongst the models depend on: 

(i) whether distinct factor structures across levels can be interpreted meaningfully, 

(ii) whether standardisation of variables is necessary, and 

(iii) whether the researcher wants to estimate second level factor variance. 

It appears from the analyses performed on the current set of empirical data, that if 
distinct factor structures are interpretable, then the isomorphic Models B and E would 
provide best fit to the data, and in most cases, these distinct models would yield no 
better fit than the i^ore parsimonious model D. Because of parsimony and ease of 
interpretation, Model D is preferred. On the other hand, while the factor loadings 
from the non scaled models are not easily interpreted, Model A with constrained 
equal loadings provides a direct estimate of school effect that is not available from 
any other models. In this sense, except the very restrictive Model C, each of the other 
Bilevel factor models provides useful and in fact complementary pieces of 
information. 

Results of the runs also indicate that the pseudo balanced estimates arc quite robust 
with regard to the extent of unbalance of the sample structure. How much of this 
robustness depends on the total sample size is discussed elsewhere (see McDonald 
and Mok, 1993 for a simulated study). Likewise, the robustness of these models with 
respect to the sizes of the intraclass correlations cannot be fully assessed without 
conducting a simulation study. 

So far discussions have been restricted to a one - factor Spearman case. Future 
research could be extended in the direction of different factor structures across levels. 
The mathematical models discussed in this paper placed no restrictions on the number 
of factors and the arguments can be extended easily to more complicated structures. 
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Items 



Common / 
Factor / 

/! School- ^\ \ 

/ level \ 


i . 

1 . The curriculum of this school 
meets my present needs. 


2. This school offers a good 
range of subjects in Years 
11 and 12. 


3. The subjects offered at this 
school develop the capacity for 
independent and critical thinking 


I Student- j ^ 
\ level /^^^ 

/ 


4. The subjects taught in this school 
offer useful knowledge or skills. 


5. The subjects taught in this 
school are relevent to real life 
and to students' needs. 


6. The subjects taught here prepare 
students adequately for future employment 



Unique Factors 



Figure 1. Model A: Equal loadings across levels; no scaling is used. 
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Items 



Unique Factors 



1 . The curriculum of litis school meets 
my present needs. 



Common 
Factor 




2. This school offers a good range of 
subjects in Years 1 1 and 1 2 . 



3. The subjects offered at this school 
develop the capacity for independent 
and critical thinking. 



4. The subjects taught in this school 
offer useful knowledge and skills. 



6. The subjects taught here prepare 
students adequately for future 
employment. 



5. The subjects taught in the school areneeds 
relevant to real life and to students' 



Common 
Factor 



Student- 



Figure 2. Model B: Unconstrained loadings across levels; no scaling is used. 
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Items 



Common 
factor 




1. The curriculum of this school mee t, 
my present needs. 

a This school offers a good range of 
subjects in Years 11 and 12. 

3. The subjects offered at this school 
develop the capacity for 
independent and critical thinking. 

4. The subjects taught in this school 
offer useful knowledge or skills. 



5. The subjects taught in the school 
are relevant to real life and the 
students' needs. 

6. The subjects taught here prepare 
students adequately for future 
employment. 



Unique 
Factors 



]*■© 



Figure 3. Random Coefficient Model 
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Table 1 . Loadings from Confirmatory Single-level Factor Analysis and Intr aclass correlations O) 



No item Stam Loading^ intra 
com 



QUAL • Quality of ttw Formal Curriculum (Chi-sq - 755.77, df = 9, McDonald's index = 0.94) 



A< 

41 


The curriculum of this school meats my present needs. 


.64 (.01) 


.04 


AC 

45 


This school offers a good range of subjects in Years 1 1 and 1 2. 


.64 (.01) 


.16 


46 


ThA £JkhlAr4<: flf farad at thift crJvv\| rtouolnn tho rjmarJtu f/v Inrtonaiviant anH nritlral think Inn 




.uo 


47 


ThA Qiihloric tannht In thlc cr-h/vJ c\Hcx ifc/afiil Lrrvm/Lvt/v> nr cVlltc 
1 IttJ oUUJtruto lauyill HI UrwSMlWI UHOT UOVlUI lUIUWRAJyU Ul SfUHo. 




no, 




i riu sHjujwio idugni in mo ocnoui are retavani 10 recu we ana 10 swoonis neeus 


act i m\ 


no 


50 


Thft cjihlArtc talinht Kara nranara ctiirlanfc artaniiatolu Ihr fiihira amnl/\um^nt 


.00 l.Ulj 


(YA 




guts • out-ot-sch9pi pumcuium fcni-sq * 58.08. or « 2. Mcuonaid s index » 0.99) 








There are opportunities for students to get to know teachers outside the classroom. 


CtZ 1 A4\ 

.55 (.01) 


.08 


A*} 

43 


The out-of-school activities of the school have sufficient variety and scope. 


.81 (.02) 


.10 


44 


There Is a good sports program in the school. 


.56 (.01) 


.26 


CO 

53 


The school places sufficient emphasis on cultural activities (music ,art f drama, etc.) 


.42 (.02) 


.09 




ATEC - Relationships with teachers (Chl-sq * 2770.29. df * 54 r McDonald's Index = 0 .80) 






56 


Most teachers are well qualified and have good teaching skills. 


.66 (.01) 


.06 


58 


Most teachers In this school show a good deal of school spirit. 


.66 (.01) 


.10 


60 


Most teachers know their Year 12 students as individual persons. 


.57 (.01) 


.04 


64 


Most teachers are part of the school community. 


.70 (.01) 


.07 


65 


Most teachers have a professional attitude towards their teaching. 


.68 (.01) 


.03 


DO 


Most teachers carry out their work with energy and pleasure. 


.76 (.01) 


.04 




Most teachers take a personal Interest in me. 


.67 (.01) 


.04 


oO 


Most teachers give students sufficient encouragement. 


.72 (.01) 


.03 


oa 
83 


if students nave difficulty with their school work, most teachers take time to help them. 


.68 (.01) 


.04 


DC 

00 


Most teachers go out their way to help you. 


72 (.01) 


.05 


OS 

00 


Most teachers show that people are more important than rules. 


.57 (.01) 


.04 


90 


Most teachers here are caring and willing to assist students who need help. 


.73 (.01) 


.04 




STDM - Student Morale (Chi-sq . 1566.18. df . 35. McDonald's Innax = 0 ftfl) 






55 


Students here think a lot of their school. 


.62 (.01) 


.15 




Everyone has a lot of fun at this school. 


.67 (.01) 


.07 


(V 


n 900a sp*rn 01 community exists amongst rear 1^ stuoents. 


.62 (.01) 


.07 


70. 


1 nave oeen nappy ai scnooi. 


.77 (.01) 


.03 


7fi 


oiuoenis ai inis sctkxh 00 not uikkj weariig uie scnooi uniform. 


.49 (.01) 


.06 


70 


MlAflfAfUt tft AO tr\ molfA iiaii (aa! o4 Kama Im tKtj* r*r*iyr\ A.J 

everyone ines 10 maKe you teei at rxxne in mfs scnooi. 


.67 (.01) 


.06 


0/ 


My experience or ints scnooi nas oeen a nappy one. 


.82 (.01) 


.03 


91 


1 am haoov to be a student at !his school 


.oh \Xf i ) 


.uo 


92 


Schonl rules here eonoitrano saII wii crSnl 1 anri rAcnnn<uhiIih/ 


.DO ^.Ul; 


f\A 


93 


There is a haoov atmosohere in the school 


7ft ( m\ 


en 




min ■ AniiuoBS to uii rnncipai, ^uosq « i4o.//, ul » o, MCLKXiaiQ S ln06X = U.yy) 






62 


1 can approach the Principal at any time for advice and help. 


.63 (.01) 


18 


71 


The Principal ensures that the school provides a good education to students. 


.70 (.01) 


.09 


81 


The Principal encourages a sense of community and belonging in the school. 


.81 (.01) 


.15 


89 


The Principal places importance on the relgious nature o f the Catholic school. 


.42 (.01) 


.10 


94 


The Principal provides good leadership of the school community. 


.81 (.01) 


.17 




ATTD - Attitudes toward Discipline W (CW-sq.24 .19. df «2 . IMlnrukfe InHftx = nQQft) 






59 


Year 1 2 students are not given sufficient real freedom here. 


.63 (.01) 


.13 


74 


This school places too much emphasis on external conformity to rules and regulations. 


.60 (.01) 


.09 


78 


Most teachers never explain why they ask you to do things around here. 


.42 (.01) 


.04 


84 


There are too many rules which restrict students' freedom. 


.78 (.01) 


.13 



Notes: 

(1) Maximum Likelihood estimates using COSAN (Fraser, 1987) for the Single level Factor analysis and Maximum Likelihood 
estimates on intra class correlations using the ML3 package (Rasfc sh, Prosser, Goldstein, 1990) were reported here. 

(2) Standard errors of factor loadings were printed ki brackets. 

(3) These items were reversely coded. 
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Table 2. Bilevel Factor Loadings and Single level factor loadings 0) 



(•) Scale: Quality of Formal Curriculum (QUAL 6 Items. Intraclass correlation s 0.08) 



Bllaval ftotutts : School lovtl 


Sinalt level 
Results 


itflm 


llodol A 

Constrained 
equal, No scaling 


Modal B 

Unconstrained, 
No scaling 


RNUH w 

Constrained 
equal, Scaling 
both levels 


UodelD 

mum v 

Constrained 
equal, Scaling at 

both tevete, 
estimating sch- 
level factor var 


Modal E 

Unconstrained, 
Scaling at both 
levels 


41 


.68 


.21 


53 


£3 


54 








(.03) 


(.01) 


(.01) 


(.03) 




45 


.70 


.41 


53 


£3 


55 








(.06) 


(.01) 


(.01) 


(.05) 




46 


.64 


2\ 


.71 


.70 


.98 








(.03) 


(.01) 


(.01) 


(.02) 




47 


.63 


.15 


.74 


.73 


55 








(.02) 


(.01) 


(.01) 


(.03) 




40 


.68 


.14 


£6 


J66 


59 








(.02) 


(.01) 


(.01) 


(.06) 




50 


.65 


.17 


£5 


£5 


57 








(.03) 


(.01) 


(.01) 


(.05) 




BilavaJ RMUItt : Student leva) 




41 


.68 


.68 


£3 


£3 


52 


54 






(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


45 


.70 


.70 


53 


£3 


53 


.64 






(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


46 


.64 


.63 


.71 


.70 


.70 


.72 






(.01) 


(.01) 


(.01) 


'.on 


(.01) 


47 


.63 


.63 


.74 


.73 


.73 


.74 






(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


49 


M 


58 


£6 


£6 


.66 


56 






(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


50 


£5 


55 


£5 


£5 


55 


56 






(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


Goodiwi of fit of modal* 


Chl-aq 
(Unbalanced 

run) 


757.1 


721.8 


768.3 


7275 


7215 


7555 


Chl-sq 
(Pwudo- 
balancad) 




7285 


7805 


734.7 


7215 




d.f. 


23 


18 


24 


23 


18 


9 


Eatimat** of proportion of variation at SchooUa val 


Estimate 


proportion of 
School-level 
factor variance^ 

0.07 


0.11 


Not Applicable 


proportion of 
School-level 
factor variance* 

0.64 


ta'BfiSB + 

0.65 





Notes: 

(1) Single level factor results were repeated here to facilitate comparisons between models 
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Table 2. Bilevel Factor Loadings and Single level factor loadings (ConU 0) 



(b) Scale: Out-nf-School Curriculum (OUTS. 4 items. Intraclaas correlation = 0.16^ 



Bllavel Results : School level 


CImmIV I .11.1 

%MO{jlt LfVM 


Rom 


Modal A 

Constrained 
equal, No scaling 


Unconstrained, 
No scaling 


Modal C 

Constrained 
equal, Scaling 
both levels 


Modal D 

Constrained 
equal, Scaling at 

both levels, 
estimating sch- 
level (actor var 


Modal E 

Unconstrained, 
Scaling at both 
levels 


42 


54 


J25 


56 


56 


.73 






(.02) 


(.05) 


(.01) 


(.01) 


(.08) 




43 


51 


.42 


.78 


.78 


1.09 






(.02) 


(.05) 


(.01) 


(.01) 


(.07) 




44 


£3 


.46 


54 


54 


56 






(.02) 


(.09) 


(.01) 


(.01) 


(.09) 




53 


.49 


.12 


.42 


A\ 


51 






(.02) 


(.06) 


(.01) 


(.01) 


(.14) 




Bilevel RmuIU : Student Leva) 




42 


£4 


.64 


56 


56 


56 


JS5 




(.02) 


(.02) 


(.01) 


(.01) 


(.01) 


(.01) 


43 


51 


50 


.78 


.78 


.78 


51 




(.02) 


(.02) 


(.01) 


(.01) 


(.01) 


(.02) 


44 


53 


£3 


54 


54 


54 


56 




(.02) 


(.02) 


(.01) 


(.01) 


(.01) 


(.01) 


S3 


.49 


.49 


.42 


A\ 


.42 


.42 




(.02) 


(.02) 


(.01) 


(.01) 


(.01) 


(.02) 


Goodness of fit of modele 


Chl-sq 
(Unbalanced 

run) 


465 




584 


39.4 


355 


68.1 


Chl-tq 
(Pseudo- 
balancod) 


495 


395 


595 


425 


395 




d.l. 


23 


18 


24 


23 


18 


9 


Estimate* of proportion of variation at School-level 


Estimate 


proportion of 
School-level 
factor variance « 

0.17 


&'b1b + 
0.20 


Not Applicable 


proportion of 
School-level 
factor variance ■ 

0.64 


(fl'BOB + 
0.62 
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Table 2. Bilevel Factor Loadin gs and Single level factor loadings (Cont.) 0) 



(c) Scale: Attitudes towards the Principal (PRIN. 5 items) Infraclass coirelatlon = 0.18) 



Bllevel RmuIU : School level 




Item 


Modal A 

Constrained 
equal, No 
scaling 


Modal B 

Unconstrained, 
No scaling 


Modal C 

Constrained 
equal, Scaling 
both levels 


Modal D 

Constrained 
equal, Scaling at 

both levels, 
estimating sen- 
level factor var 


Model E 

Unconstrained 
.Scaling at 
both levels 


Single Laval 
Reaulta 


62 


.76 


.42 


.61 


.60 


.73 






(.02) 


(.07) 


(.01) 


(.01) 


(.07) 




71 


.68 


.28 


M 


J68 


M 






(-01) 


(04) 


(.01) 


(01) . 


(.03) 




81 


.85 


.45 


.78 


.78 


M 






(.01) 


(.05) 


(.01) 


(.01) 


(.02) 




89 


.40 


.14 


.42 


A2 


.43 






(.01) 


(.05) 


(.01) 


(.01) 


(.13) 




94 


M 


.50 


.78 


.77 


.95 






(.01) 


(.06) 


(.01) 


(.01) 


(.02) 




Bllevel Reeuite : Student Laval 




62 


.76 


.76 


.61 


£0 


£1 


£3 




(.02) 


_ .. .(-02) 


(.01) 


(.01) 


(.01) 


. (01) 


71 


.68 


£8 


AB 


.68 


.68 


.70 




(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


81 


.85 


.85 


.78 


.78 


.77 


.81 




(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


89 


.40 


.40 


.42 


.42 


42 


.43 




(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


94 


.90 


.89 


.78 


.77 


.78 


31 




(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


(.01) 


Goodneaa of fit of modele 


Chl-aq 
(Unbalancad 

run) 


125.6 


1116 


162.4 


1175 


11Z6 


1488 


Chl-aq 
(Peeudo- 
balanced) 


130.9 


119.9 


1744 


123J 


1194 




d.f. 


23 


18 


24 


23 


18 


9 


Eatlmates of proportion of variation at School-level 




Estimate 


proportion of 
School-level 
factor variance r 

0.21 


(&'B*e + 
0.21 


Not Applicable 


proportion of 
School-level 
factor variance x 

0.61 


(a'BOB + 

fl'w^w) - 
0.21 
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Table 2. Bllevel Factor Loadin gs and Single level factor loadings (Cont.) (1) 



' <) Seal* : AttitudM toward Dlaclpllne (ATTD, 4 item*, Intraclaee correlation ■ 0.16) 



Bllevel Results : School Level 




Ram 


ModelA 

Constrained 
equal, No 
scaling 


Modal B 

Unconstrained, 
No scaling 


Model C 

Constrained 
equal, Scaling 
Dotn levels 


Modal D 

Constrained 
equal, Scaling 
at Dotn levels, 
esumasng sen* 

LauaI for*! /v ii or 
KJVB1 RaCKX Vol 


Model E 

Unconstrained, 
Scaling at both 
levets 


Single level 
Results 


M 


• 99 


Aft 


to 
.00 


CO 
•So 


OA 








l.UOJ 


(An) 


l.uij 






74 


•V 1 


11 


.57 

fVf 




01 






102) 


VV?/ 






l.UOJ 




78 


.43 


.14 


.42 


,42 








(.02) 


(.03) 


(.01) 


(01) , 


(.10) 




84 


XI 


.47 


.74 


.72 


1.02 






(.02) 


(.05) 


(.01) 


(.01) 


(.01) 








Bilevel Results : 


Student Level 








59 


.75 


.74 




58 


58 


.63 




(.02) 


(.02) 


(.01) 


(.01) 


f.01* 


(.01) 


74 


&\ 


.61 


SI \ 


57 


J56 






(02) 


(02) 


(.01) 


(.01) 


(.01) 


(.01) 


78 


.43 


.45 


.42 


.42 


.42 


42 




(.02) 


(.02) 


(X>1) 


(.01) 


(.01) 


(.01) 


84 


.87 


X! 


.74 


.72 


.73 


.78 




(.02) 


(.02) 


(.0^ 


(.01) 


(.01) 


(.01) 


Goodness of fit of models 


Chl-sq 
(Unbalanced 
run) 


41.2 




117.9 


m 


232 


24.1 


Chl-sq 
(Pseudo- 
balancad) 


43.0 


24.3 


123J» 


34.1 


24J 




If. 


23 


18 


24 


23 


18 


9 


Estimate* of proportion of variation at School-level 


Estimate 


proportion of 
School-level 
(actor variance * 




Not Applicable 


proportion of 
School-level 
factor variance 

a 








023 


023 




0.68 


0,70 
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